Evaluating statistical language models as pragmatic reasoners

The relationship between communicated language and intended meaning is often probabilistic and sensitive to context. Numerous strategies attempt to estimate such a mapping, often leveraging recursive Bayesian models of communication. In parallel, large language models (LLMs) have been increasingly applied to semantic parsing applications, tasked with inferring logical representations from natural language. While existing LLM explorations have been largely restricted to literal language use, in this work, we evaluate the capacity of LLMs to infer the meanings of pragmatic utterances. Specifically, we explore the case of threshold estimation on the gradable adjective ``strong'', contextually conditioned on a strength prior, then extended to composition with qualification, negation, polarity inversion, and class comparison. We find that LLMs can derive context-grounded, human-like distributions over the interpretations of several complex pragmatic utterances, yet struggle composing with negation. These results inform the inferential capacity of statistical language models, and their use in pragmatic and semantic parsing applications. All corresponding code is made publicly available (https://github.com/benlipkin/probsem/tree/CogSci2023).Comment: 8 pages, 4 figures, to appear in the Proceedings of the Annual Meeting of the Cognitive Science Society 202

LILO: Learning Interpretable Libraries by Compressing and Documenting Code

While large language models (LLMs) now excel at code generation, a key aspect of software development is the art of refactoring: consolidating code into libraries of reusable and readable programs. In this paper, we introduce LILO, a neurosymbolic framework that iteratively synthesizes, compresses, and documents code to build libraries tailored to particular problem domains. LILO combines LLM-guided program synthesis with recent algorithmic advances in automated refactoring from Stitch: a symbolic compression system that efficiently identifies optimal lambda abstractions across large code corpora. To make these abstractions interpretable, we introduce an auto-documentation (AutoDoc) procedure that infers natural language names and docstrings based on contextual examples of usage. In addition to improving human readability, we find that AutoDoc boosts performance by helping LILO's synthesizer to interpret and deploy learned abstractions. We evaluate LILO on three inductive program synthesis benchmarks for string editing, scene reasoning, and graphics composition. Compared to existing neural and symbolic methods - including the state-of-the-art library learning algorithm DreamCoder - LILO solves more complex tasks and learns richer libraries that are grounded in linguistic knowledge

From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought

How does language inform our downstream thinking? In particular, how do humans make meaning from language -- and how can we leverage a theory of linguistic meaning to build machines that think in more human-like ways? In this paper, we propose \textit{rational meaning construction}, a computational framework for language-informed thinking that combines neural models of language with probabilistic models for rational inference. We frame linguistic meaning as a context-sensitive mapping from natural language into a \textit{probabilistic language of thought} (PLoT) -- a general-purpose symbolic substrate for probabilistic, generative world modeling. Our architecture integrates two powerful computational tools that have not previously come together: we model thinking with \textit{probabilistic programs}, an expressive representation for flexible commonsense reasoning; and we model meaning construction with \textit{large language models} (LLMs), which support broad-coverage translation from natural language utterances to code expressions in a probabilistic programming language. We illustrate our framework in action through examples covering four core domains from cognitive science: probabilistic reasoning, logical and relational reasoning, visual and physical reasoning, and social reasoning about agents and their plans. In each, we show that LLMs can generate context-sensitive translations that capture pragmatically-appropriate linguistic meanings, while Bayesian inference with the generated programs supports coherent and robust commonsense reasoning. We extend our framework to integrate cognitively-motivated symbolic modules to provide a unified commonsense thinking interface from language. Finally, we explore how language can drive the construction of world models themselves

The Neuro-Symbolic Inverse Planning Engine (NIPE): Modeling Probabilistic Social Inferences from Linguistic Inputs

Human beings are social creatures. We routinely reason about other agents, and a crucial component of this social reasoning is inferring people's goals as we learn about their actions. In many settings, we can perform intuitive but reliable goal inference from language descriptions of agents, actions, and the background environments. In this paper, we study this process of language driving and influencing social reasoning in a probabilistic goal inference domain. We propose a neuro-symbolic model that carries out goal inference from linguistic inputs of agent scenarios. The "neuro" part is a large language model (LLM) that translates language descriptions to code representations, and the "symbolic" part is a Bayesian inverse planning engine. To test our model, we design and run a human experiment on a linguistic goal inference task. Our model closely matches human response patterns and better predicts human judgements than using an LLM alone.Comment: To appear at ICML Workshop on Theory of Mind in Communicating Agent

Antimicrobial use Guidelines for Treatment of Respiratory Tract Disease in Dogs and Cats: Antimicrobial Guidelines Working Group of the International Society for Companion Animal Infectious Diseases

Respiratory tract disease can be associated with primary or secondary bacterial infections in dogs and cats and is a common reason for use and potential misuse, improper use, and overuse of antimicrobials. There is a lack of comprehensive treatment guidelines such as those that are available for human medicine. Accordingly, the International Society for Companion Animal Infectious Diseases convened a Working Group of clinical microbiologists, pharmacologists, and internists to share experiences, examine scientific data, review clinical trials, and develop these guidelines to assist veterinarians in making antimicrobial treatment choices for use in the management of bacterial respiratory diseases in dogs and cats.M.R. Lappin, J. Blondeau, D. Boothe, E.B. Breitschwerdt, L. Guardabassi, D.H. Lloyd, M.G. Papich, S.C. Rankin, J.E. Sykes, J. Turnidge, and J.S. Wees

Impact of Sequencing Targeted Therapies With High-dose Interleukin-2 Immunotherapy: An Analysis of Outcome and Survival of Patients With Metastatic Renal Cell Carcinoma From an On-going Observational IL-2 Clinical Trial: PROCLAIM

BACKGROUND: This analysis describes the outcome for patients who received targeted therapy (TT) prior to or following high-dose interleukin-2 (HD IL-2). PATIENTS AND METHODS: Patients with renal cell carcinoma (n = 352) receiving HD IL-2 were enrolled in Proleukin RESULTS: Overall, there were 4% complete response (CR), 13% partial response (PR), 39% stable disease (SD), and 43% progressive disease (PD) with HD IL-2. The median overall survival (mOS) was not reached in patients with CR, PR, or SD, and was 15.5 months in patients with PD (median follow-up, 21 months). Sixty-one patients had prior TT before HD IL-2 with an overall response rate (ORR) to HD IL-2 of 19% (1 CR, 9 PR) and an mOS of 22.1 months. One hundred forty-nine patients received TT only after HD IL-2 with an mOS of 35.5 months. One hundred forty-two patients had no TT before or after HD IL-2, and mOS was not reached. The mOS was 8.5 months in PD patients who received HD IL-2 without follow-on TT and 29.7 months in PD patients who received follow-on TT after HD IL-2. CONCLUSIONS: HD IL-2 as sole front-line therapy, in the absence of added TT, shows extended clinical benefit (CR, PR, and SD). Patients with PD after HD IL-2 appear to benefit from follow-on TT. Patients who progressed on TT and received follow-on HD IL-2 experienced major clinical benefit. HD IL-2 therapy should be considered in eligible patients

The Integrin Antagonist Cilengitide Activates αVβ3, Disrupts VE-Cadherin Localization at Cell Junctions and Enhances Permeability in Endothelial Cells

Cilengitide is a high-affinity cyclic pentapeptdic αV integrin antagonist previously reported to suppress angiogenesis by inducing anoikis of endothelial cells adhering through αVβ3/αVβ5 integrins. Angiogenic endothelial cells express multiple integrins, in particular those of the β1 family, and little is known on the effect of cilengitide on endothelial cells expressing αVβ3 but adhering through β1 integrins. Through morphological, biochemical, pharmacological and functional approaches we investigated the effect of cilengitide on αVβ3-expressing human umbilical vein endothelial cells (HUVEC) cultured on the β1 ligands fibronectin and collagen I. We show that cilengitide activated cell surface αVβ3, stimulated phosphorylation of FAK (Y397 and Y576/577), Src (S418) and VE-cadherin (Y658 and Y731), redistributed αVβ3 at the cell periphery, caused disappearance of VE-cadherin from cellular junctions, increased the permeability of HUVEC monolayers and detached HUVEC adhering on low-density β1 integrin ligands. Pharmacological inhibition of Src kinase activity fully prevented cilengitide-induced phosphorylation of Src, FAK and VE-cadherin, and redistribution of αVβ3 and VE-cadherin and partially prevented increased permeability, but did not prevent HUVEC detachment from low-density matrices. Taken together, these observations reveal a previously unreported effect of cilengitide on endothelial cells namely its ability to elicit signaling events disrupting VE-cadherin localization at cellular contacts and to increase endothelial monolayer permeability. These effects are potentially relevant to the clinical use of cilengitide as anticancer agent

Analysis of Temperature-to-Polarization Leakage in BICEP3 and Keck CMB Data from 2016 to 2018

The Bicep/Keck Array experiment is a series of small-aperture refracting telescopes observing degree-scale Cosmic Microwave Background polarization from the South Pole in search of a primordial B-mode signature. As a pair differencing experiment, an important systematic that must be controlled is the differential beam response between the co-located, orthogonally polarized detectors. We use high-fidelity, in-situ measurements of the beam response to estimate the temperature-to-polarization (T → P) leakage in our latest data including observations from 2016 through 2018. This includes three years of Bicep3 observing at 95 GHz, and multifrequency data from Keck Array. Here we present band-averaged far-field beam maps, differential beam mismatch, and residual beam power (after filtering out the leading difference modes via deprojection) for these receivers. We show preliminary results of "beam map simulations," which use these beam maps to observe a simulated temperature (no Q/U) sky to estimate T → P leakage in our real data

Observing low elevation sky and the CMB Cold Spot with BICEP3 at the South Pole

BICEP3 is a 520 mm aperture on-axis refracting telescope at the South Pole, which observes the polarization of the cosmic microwave background (CMB) at 95 GHz to search for the B-mode signal from inflationary gravitational waves. In addition to this main target, we have developed a low-elevation observation strategy to extend coverage of the Southern sky at the South Pole, where BICEP3 can quickly achieve degree-scale E-mode measurements over a large area. An interesting E-mode measurement is probing a potential polarization anomaly around the CMB Cold Spot. During the austral summer seasons of 2018-19 and 2019-20, BICEP3 observed the sky with a flat mirror to redirect the beams to various low elevation ranges. The preliminary data analysis shows degree-scale E-modes measured with high signal-to-noise ratio